Tech Arsenal 1/Tech Arsenal (Arsenal Computer).ISO/tek-02/objovr.zip/OBJOVR.DOC

Text File | 1993-01-04 | 15KB | 436 lines

David Dubois [71401,747]
Zelkop Software
1988.10.18

01/10/90  Fixed bug in the "overlay detector"
          Ron Schuster [76666,2322]

OBJOVR
======

Description of a technique for accessing "OBJitized" data within an
overlayed unit. By OBJitized data, I mean data which has been
converted to an .OBJ file using the BinObj utility.

The Turbo Pascal documentation shows how to use BinObj to link data
directly into your program. It also suggests that such data shouldn't
be overlayed. Here, I will show how to get around that limitation. If
you haven't read the documentation about BinObj, you should do so
before reading this.

I will demonstrate the technique through a simple example.

Files
=====

  ObjOvr.Doc  This documentation
 MakeOBJ.Pas  Program to generate example .OBJ file
 LookOBJ.Pas  The unit with OBJitized data
  UseOBJ.Pas  The program that uses the unit
OBJTypes.Pas  A supporting unit
InitOver.Pas  Another supporting unit

How to run the example.
=======================

First, compile and run MakeOBJ. This creates the .OBJ file that will
be linked in. It uses the DOS Exec routine to make the appropriate
call to BinObj. You may have to change the path in the EXEC call.

Then, compile and run UseOBJ. Since UseOBJ contains an overlayed file,
it must be compiled to disk. Upon running the program, you see that
unit LookOBJ has detected that it is overlayed. The program then
prints out the OBJitized data, which happens to be the square roots of
the numbers 1 through 100.

The program will work whether or not the unit is overlayed. To see
this, insert a space between "{" and "$O LookOBJ}". This will cause
the overlay directive to be ignored. Compile and run the program
again. Neither a Make nor Build will be necessary. Note that the
LookOBJ unit "knows" that it is no longer overlayed.

What's the problem?
===================

Why is it necessary to do anything fancy? Why is referencing OBJitized
data in an overlayed unit any different? First, you have to understand
a little about how the overlay manager works.

An overlayed unit lives a double life. On the one hand, it consists of
code that is relocatable. That is, each time a routine is called, it
may be in a different place in memory. On the other hand, other
routines have to be able to find the overlayed routines in order to
call them, even if they aren't currently loaded in memory.

In effect an overlayed unit has two different segments. One of these
segments is small and permanent. This segment is part of the .EXE
file. Once the .EXE file is loaded, this segment doesn't move. It
consists of small chunk of code for each public routine in the
overlayed unit plus a little overhead. This is the segment that is
shown in a .MAP file.

The routines themselves will be in another segment. This segment may
be anywhere and may change as the overlay manager loads and re-loads
the unit. Of course, if the unit is not currently loaded, then it is
in no segment at all.

Whenever you make a call to a routine in the overlayed unit, or refer
to its address, (either by using @, or by storing it in a procedural
variable, or passing it as a parameter), you are referring to first,
permanent segment. The code there, in turn, refers to the actual code,
should it be loaded.

So, what happens when you use @ to refer to data that has been
OBJitized? Let's look at an example.

        procedure DataLink;
        external;
        {$L DataLink.OBJ}

What is the value of "@ DataLink"? Well, it depends on whether it
resides in an overlayed unit. If the unit is not overlayed, then
@ DataLink will be the address of the actual data. If the unit is
overlayed, it will be the address of a small chunk of code in the
permanent segment. In order to find the data, we have to know a little
more about that chunk.

How do you find the data?
=========================

That chunk will be one of two things. When the overlayed unit is not
currently loaded, it will consist of an interrupt call, INT $3F.
Interrupt $3F is handled by the overlay manager. When the unit is
loaded, those interrupt calls are replaced with FAR CALLs to the
appropriate place in memory. The far-called segment will depend on
where the unit was loaded. When the unit is un-loaded, the interrupt
calls re-appear.

So in our example above, if the unit is overlayed, and the unit has
been loaded, then @ DataLink is the address of the FAR CALL. That
call, in turn, points to the data that we are looking for.

Please note that this depends on whether the unit is actually
overlayed, regardless of whether it was compiled with the $O+
(overlays allowed) option activated. If the unit has been compiled so
that it CAN be overlayed, (i.e., $O+ is on), but is not ACTUALLY
overlayed, (i.e. the {$O UnitName} directive was not given), then @
DataLink will refer to the actual data.

Therefore, in order to decide how to handle itself, a unit has to be
able to determine whether it has been overlayed. This can be
determined by looking at offset 0 of the unit's segment. If the unit
is overlayed, then the segment will be that special permanent segment
that I have referred to. It so happens that, if the segment is
overlayed, that will always begin with an INT $3F instruction.  Therefore,
if you consider that to be a Word, the number $3FCD is stored there.

Note that this is not fool-proof. It is possible that a non-overlayed
unit may have that special number at that particular location. This
would occur in the unlikely case that the program had range-checking
turned on, and the first range declared in the unit had a lower bound
having $3FCD as its least significant 16-bits.  This would seem unlikely
unless a malicious attempt is made by the programmer to confuse the
"overlay detector".

[David's original "overlay detector" also checked whether the SECOND
word of the segment was equal to zero.  Although word usually appears
to contain a zero, it is not always.  This would cause strange and
difficult-to-find bugs.  See the file OVRLAY.TXT (CompuServe BPROGA
forum LIB 2) for more details. -- RJS]

So, with this knowledge well in hand, we are ready to face the
example. Here is the code with detailed documentation:

Details, details
================

OBJTypes
--------

{For the purposes of this example, the OBJitized data will consist of
an array of 100 reals. This type must be accessed from several places,
so I put it into a unit by itself.}

unit ObjTypes;

interface

  const
    Max = 100;
  type
    OBJitizedDataType = array [ 1 .. Max ] of real;

implementation

end.

MakeOBJ
-------

{MakeOBJ is a stand-alone program that generates the .OBJ file that
will be linked into our example program. The data consists of the
square roots of the numbers 1 through 100.}

{MakeOBJ is going to make a call to the DOS Exec routine, so, we must
make sure we leave room for our spawned program to run. Here we
allocate the minimum stack size, and no heap.}

{$M 1024,0,0}

{DOS provides access to the Exec routine, while OBJTypes provides
access to OBJitizedDataType, (as well as Max.)}

uses
  DOS, OBJTypes;

{We will fill OBJitizedData with the data we want to OBJitize. Then
we will write it to BinaryDataFile.}

var
  OBJitizedData  : OBJitizedDataType;
  BinaryDataFile : file of OBJitizedDataType;
  I              : integer;

begin
  writeln ( 'MakeOBJ' );
  writeln;
  writeln ( 'Generating data to OBJitize ...' );

  for I := 1 to Max do
    OBJitizedData [ I ] := sqrt ( I );

  writeln ( 'Creating data to file ...' );

  assign  ( BinaryDataFile, 'OBJitize.DAT' );
  rewrite ( BinaryDataFile                 );
  write   ( BinaryDataFile, OBJitizedData  );
  close   ( BinaryDataFile                 );

  writeln ( 'Executing BinOBJ ...' );

  {The code above created the file OBJitize.DAT. The BinObj utility
  will be used to convert that to an .OBJ file that can be linked into
  a program. OBJitizedDataLink is the identifier that must be used in
  the program. This assumes that BinOBJ will be found in the path
  "c:\TP\". If yours is elsewhere, you will have to change this.}

  Exec ( 'c:\TP\BinOBJ.EXE', 'OBJitize.DAT OBJitize.OBJ OBJitizedDataLink' );
end.

InitOver
--------

{Our example unit will have an initialization section. Therefore, I
have to have a unit that will initialize the overlay manager, before
the example unit is initialized. See the Turbo Pascal documentation
referring to the standard Overlay unit.}

unit InitOver;

interface

  uses
    Overlay;

implementation

begin
  OvrInit ( 'UseOBJ.OVR' );
end.

LookOBJ
-------

{We will turn on the overlays-allowed switch. This will not force the
unit to be overlayed, but makes it possible. }

{$O+}

unit LookOBJ;

interface

  {This function returns the Ith component of our example data. So
  calling OBJitized ( 16 ) will return the 16th component of our
  array, which happens to be 4.0.}

  function OBJitized ( I : integer ) : real;

implementation

uses
  OBJTypes;

{This unit will detect whether or not it has been overlayed. It stores
the result here.}

var
  ThisUnitIsOverlayed : boolean;

{For the purposes of using this technique, we will require a pointer
to the data, as well as a pointer to a pointer to the data.}

type
  OBJitizedDataPtr    = ^ OBJitizedDataType;
  OBJitizedDataPtrPtr = ^ OBJitizedDataPtr;

  {Here is the OBJitizedData.}

  procedure OBJitizedDataLink;
  external;
  {$L OBJitize.OBJ }

  {Here is the procedure that figures out where the OBJitized data is.
  It returns a pointer to the data. It assumes that
  ThisUnitIsOverlayed has already been set correctly. If the unit has
  not been overlayed, then finding the data is a simple task. Just
  take the address of OBJitizedDataLink. If the unit is overlayed,
  then we havfe a tougher problem. In this case

                         @ OBJitizedDataLink

  is not a pointer to the data at all. Instead, it a pointer to a FAR
  CALL. The far call consists of one byte, an $EA as it happens,
  followed by a four-byte segment-offset pair. That address is the
  address of the data we are looking for. To skip over the one byte, I
  convert the address into a number, add one, and then convert it back
  into a pointer again.

                    longint ( @ OBJitizedDataLink )
               succ ( longint ( @ OBJitizedDataLink ) )
    OBJitizedDataPtrPtr ( succ ( longint ( @ OBJitizedDataLink ) ) )

  That points to the operand of the far call. When de-referenced, we
  have the pointer to the data.

  Note, that will only work if the unit containing OBJitiziedDataLink
  is currently loaded. We know that it loaded since this function
  occupies the same unit.}

  function OBJitizedData : OBJitizedDataPtr;
  begin
    if ThisUnitIsOverlayed then
      OBJitizedData := OBJitizedDataPtrPtr (
                         succ ( longint ( @ OBJitizedDataLink ) ) ) ^
    else
      OBJitizedData := @ OBJitizedDataLink;
  end;

  {Here is the function that refers to the data. It uses OBJitizedData
  to find the data, and simply de-references the pointer it returns to
  find the data from the .OBJ file. In this case, it returns a
  component from the array.}

  function OBJitized ( I : integer ) : real;
  begin
    OBJitized := OBJitizedData ^ [ I ];
  end;

{Here is the intialization code. This is where the unit finds out
whether it has been overlayed or not. Here we have to find offset 0 of
that small, permanent segment that refers to this unit. To do that, I
start with the address of OBJitizedDataLink.

                        @ OBJitizedDataLink

This could have been any routine in the unit. I clear the offset
portion of this address by converting it a longint, masking out the
offset using a logical and, then converting it back to a pointer
again.

                  longint ( @ OBJitizedDataLink )
             $FFFF0000 and longint ( @ OBJitizedDataLink ))
       WordPtr ( $FFFF0000 and longint ( @ OBJitizedDataLink ) )

De-referencing that pointer gives the first 2 bytes of that segment.
If the unit was overlayed that should be the longint $3FCD, which
is an INT $3F instruction.}

type
  WordPtr = ^ word;
begin
  ThisUnitIsOverlayed :=   WordPtr (    $FFFF0000
                                  and longint ( @ OBJitizedDataLink ) ) ^
                         = $3FCD;

  {For the purposes of this example, the result is reported.}

  writeln ( 'This unit is overlayed: ', ThisUnitIsOverlayed );
end.


UseOBJ
------

{Finally, the program that puts it all together.

Far calls should be forced on when using overlays.}

{$F+}

program UseOBJ;

{InitOver allows the overlay manager to be initialized before LookOBJ
is. OBJTypes provides access to Max. LookOBJ does all the work.}

uses
  InitOver, OBJTypes, LookOBJ;

{We will overlay LookOBJ. You can disable this directive by placing a
space before the dollar sign.}

{$O LookOBJ }

{The program simply writes out the values stored in the OBJitized data
by making calls to the LookOBJ unit.}

var
  I : integer;
begin
  for I := 1 to Max do
    write ( OBJitized ( I ) : 16 : 10 );
end.

Notes
=====

The code that LookOBJ uses to determine whether it is in a unit can be
moved out of the initialization section, and into the OBJitizedData
routine. This will save code and data, and forego the confusion
related to initalizing overlayed segments. But then, it will have to
be run each time the data is accessed.

The code that determines the address of the data, in OBJitizedData,
CANNOT be put into the initialization section. Since the data is
relocatable, it may theoretically be in a different place each time it
is accessed, and so its address must found anew each time.

The code for finding the address of the data, and the code that
accesses the data using that address, must be in the same unit as the
data. It would be pointless to find the address of the data, and then
un-load the data in order to load the code that accesses it. Remember,
once control returns outside the unit in which the data resides, all
bets are off.

I am
====

David Neal Dubois

You can send mail to:

Zelkop Software
P.O. Box 5177
Armdale, Nova Scotia
Canada
B3L 4M7

My CompuServe User ID is [71401,747]. You can EasyPlex me, or leave a
message on the BPROGA Borland's programmer's forum.

I am keen to receive any comments or suggestions.

I am releasing this knowledge to the public domain, but if you make
use of it, please mention my name.